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DETAILED ACTION 

Response to Arguments 

1. Applicant's arguments with respect to independent claims 1, 7, and 13 have been 
considered but are moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC § 101 

2. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

3. Claims 1-6 and 19 are rejected under 35 U.S.C. 101 because the claimed 
invention is directed to a non-statutory subject matter as follows. 

Claim 1 recites an XML-based tag for a visual cue associated to a visual 
element of an XML-based multimedia presentation, wherein the multimedia presentation 
is structured for display on a display surface of a computer wherein the XML-based tag 
comprises which is not a process, machine, manufacture or composition of matter and 
is thus non-statutory. 

Claim 2 recites an XML-based tag for a visual cue associated to a visual element 
of an XML-based multimedia presentation, wherein the multimedia presentation is 
structured for display on a display surface of a computer wherein the XML-based tag 
comprises which is not a process, machine, manufacture or composition of matter and 
is thus non-statutory. 

Claim 3 recites an XML-based tag for a visual cue associated to a visual element 
of an XML-based multimedia presentation, wherein the multimedia presentation is 
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structured for display on a display surface of a computer wherein the XML-based tag 
comprises which is not a process, machine, manufacture or composition of matter and 
is thus non-statutory. 

Claim 4 recites an XML-based tag for a visual cue associated to a visual element 
of an XML-based multimedia presentation, wherein the multimedia presentation is 
structured for display on a display surface of a computer wherein the XML-based tag 
comprises which is not a process, machine, manufacture or composition of matter and 
is thus non-statutory. 

Claim 5 recites an XML-based tag for a visual cue associated to a visual element 
of an XML-based multimedia presentation, wherein the multimedia presentation is 
structured for display on a display surface of a computer wherein the XML-based tag 
comprises which is not a process, machine, manufacture or composition of matter and 
is thus non-statutory. 

Claim 6 recites an XML-based tag for a visual cue associated to a visual element 
of an XML-based multimedia presentation, wherein the multimedia presentation is 
structured for display on a display surface of a computer wherein the XML-based tag 
comprises which is not a process, machine, manufacture or composition of matter and 
is thus non-statutory. 

Claim 19 recites an XML-based tag for a visual cue associated to a visual 
element of an XML-based multimedia presentation, wherein the multimedia presentation 
is structured for display on a display surface of a computer wherein the XML-based tag 
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comprises which is not a process, machine, manufacture or composition of matter and 
is thus non-statutory. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 1-21 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Stephen Bugaj et al in the publication titled Synchronized Multimedia Integration 
Language, November 1997, edited by Phillip Hoscha (09/09/1997) in view of Hickman 
US Patent Publication 2007/0294334, in view of Toklu et al US Patent 6,724,915 
(hereafter referenced as Bugaj, Hickman and Toklu). 

Regarding claim 1, Bugaj in section 2, 1-4 discloses SMIL (synchronized media 
integration language) for integrating a set of independent multimedia objects into a 
synchronized multimedia presentation such as slide show synchronized with audio 
comments or a video synchronized with text stream, in section 3, 1 Bugaj discloses that 
a SMIL document is an XML-based document, in section 4 Bugaj disclose the general 
syntax of a SMIL document comprising a header and a body wherein both parts contain 
elements and attributes which reads on "an XML-based tag for a visual cue associated 
to a visual element of an XML-based multimedia presentation, wherein the multimedia 
presentation is structured for display on a display surface of a computer wherein the 
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XML-based tag comprises : an element attribute that defines the visual representation of 
the visua l cue". 

In section 5, General Semantics, 1-6 Bugaj disclose a layout section of a SMIL 
document including alternative layout elements embedded in a switch element used to 
determine the placement of the presentation which reads on "attribute that defines the 
spatial characteristics of the visual cue", in section 6, General Semantics, 1 Bugaj 
discloses a schedule element which determines the temporal behavior of the SMIL 
document which reads on "and an element attribute that defines temporal 
characteristics of the visual cue". In section 7.4 page 27 Bugaj discloses a smile 
document for the newscast presentation illustrated in figure 7.1 page 26 wherein the 
layout and temporal elements are controlled by their associated attributes which reads 
on " wherein the defined temporal and spatial characteristics of the visual cue are 
relative to the temporal and spatial characteristics of the associated visual element. " 

Bugaj is silent about wherein the computer superimposes the visual cue's display 
is the associated visual element in the multimedia presentation with a visual 
appearance based on the defined visual representation of the visual cue, during a 
period of time based on the defined temporal characteristics of the visual cue : and d ata 
location over the associated visual element based on the defined spatial characteristics 
of the visual cue. 

Hickman teaches displaying an icon on a rectangular page display area on the 
monitor of a client computer which meets "wherein the computer superimposes the 
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visual cue's display is the associated visual element in the multimedia presentation" 
(Para. 0051) as shown in (figure 3. labels 74, & 80). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj to include wherein the computer 
superimposes the visual cue's display is the associated visual element in the multimedia 
presentation as taught by Hickman in order to a client computer user to access, 
diagnose and fix problems on a host computer remotely. 

Bugaj and Hickman are silent about with a visual appearance based on the 
defined visual representation of the visual cue, during a period of time based on the 
defined temporal characteristics of the visual cue : and d ata location over the associated 
visual element based on the defined spatial characteristics of the visual cue. 

Toklu teaches tracking an object in an image sequence in between two time 
instances which meets "with a visual appearance based on the defined visual 
representation of the visual cue, during a period of time based on the defined temporal 
characteristics of the visual cue " (column 11, 19-21: Toklu further teaches using the 
location and shape of the object to track the object between the given time instances 
which meets "and d ata location over the associated visual element based on the 
defined spatial characteristics of the visual cue" (column 11, 21-22). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj and Hickman to include with a visual 
appearance based on the defined visual representation of the visual cue, during a 
period of time based on the defined temporal characteristics of the visual cue : and d ata 
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location over the associated visual element based on the defined spatial characteristics 
of the visual cue as taught by Toklu in order to provide a real-time approach of tracking 
objects in a complex environment (column 1 1 , 22-24). 

Regarding Claim 2, Bugaj in section 6.1, General Semantics, 3 discloses 
schedule elements including begin and end times and in section 6.2 page 8, 3 Bugaj 
discloses duration as the difference between the end times and begin time of an 
element, which reads on "An XML-based tag as defined further wherein temporal 
characteristics include begin time, end time, and duration." 

Regarding Claim 3, Bugaj in section 6.4, Syntax discloses an image element 
tagged as an XML media object element. 

Bugaj and Hickman are silent about wherein visual representation includes color. 
Toklu teaches color intensity inside the boundary of an object which meets "wherein 
visual representation includes color" (column 6, 39-40). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj and Hickman to include wherein visual 
representation includes color as taught by Toklu in order to provide a real-time 
approach of tracking objects in a complex environment (column 11, 22-24). 

Regarding Claim 4, Bugaj in section 6.4, Syntax discloses an image element 
which reads on "An XML-based tag as defined wherein visual representation includes 
shape." 

Regarding Claim 5, in section 5, General Semantics, 1-6 Bugaj disclose a layout 
section of a SMIL document including alternative layout elements embedded in a switch 
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element used to determine the placement of the presentation which reads on "An XML- 
based tag as defined wherein spatial characteristics include position." 

Regarding Claim 6, in section 7.4 page 27 Bugaj discloses a smile document for 
the newscast presentation illustrated in figure 7.1 page 26 wherein the layout and 
temporal elements are controlled by their associated attributes in hierarchal order which 
reads on "An XML-based tag as defined wherein the XML-based tag for the visual cue 
is nested within an XML-based element that defines the associated visual element." 

Regarding claim 7, Bugaj in section 7.4, page 26, 2-9 discloses a news 
broadcast on the web as shown in figure 7.1 to the left and right which reads on "In an 
XML-based browser that displays synchronized multimedia presentations to user a 
method for processing an XML-based tag for visual cues associated with a multimedia 
element comprising:" 

On page 27 Bugaj discloses an XML data structure holding the elements of the of 
the newscast scenario written in XML code comprising elements stored in tags wherein 
each element has its own properly defined attribute including spatial and temporal 
characteristics to perform a presentation which reads on "storing information from the 
tag concerning the multimedia element to which the visual cue is associated, together 
with the information from the tag concerning visual representation and spatial and 
temporal characteristics of the visual cue; and in synchronization with display of the 
multimedia element, displaying the visual cue with the visual representation specified, 
and in the spatial and temporal relationships specified by the spatial and temporal 
characteristics." 
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In section 7.4 page 27 Bugaj discloses a smile document for the newscast 
presentation illustrated in figure 7.1 page 26 wherein the layout and temporal elements 
are controlled by their associated attributes which reads on " wherein the defined 
temporal and spatial characteristics of the visual cue are relative to the temporal and 
spatial characteristics of the associated visual element. " 

Bugaj is silent about wherein the computer superimposes the visual cue's display 
is the associated visual element in the multimedia presentation with a visual 
appearance based on the defined visual representation of the visual cue, during a 
period of time based on the defined temporal characteristics of the visual cue : and d ata 
location over the associated visual element based on the defined spatial characteristics 
of the visual cue. 

Hickman teaches displaying an icon on a rectangular page display area on the 
monitor of a client computer which meets "wherein the computer superimposes the 
visual cue's display is the associated visual element in the multimedia presentation" 
(Para. 0051) as shown in (figure 3. labels 74, & 80). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj to include wherein the computer 
superimposes the visual cue's display is the associated visual element in the multimedia 
presentation as taught by Hickman in order to a client computer user to access, 
diagnose and fix problems on a host computer remotely. 

Bugaj and Hickman are silent about with a visual appearance based on the 
defined visual representation of the visual cue, during a period of time based on the 



Application/Control Number: 10/637,625 Page 10 

Art Unit: 2423 

defined temporal characteristics of the visual cue : and d ata location over the associated 
visual element based on the defined spatial characteristics of the visual cue. 

Toklu teaches tracking an object in an image sequence in between two time 
instances which meets "with a visual appearance based on the defined visual 
representation of the visual cue, during a period of time based on the defined temporal 
characteristics of the visual cue " (column 11, 19-21: Toklu further teaches using the 
location and shape of the object to track the object between the given time instances 
which meets "and d ata location over the associated visual element based on the 
defined spatial characteristics of the visual cue" (column 11, 21-22). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj and Hickman to include with a visual 
appearance based on the defined visual representation of the visual cue, during a 
period of time based on the defined temporal characteristics of the visual cue : and d ata 
location over the associated visual element based on the defined spatial characteristics 
of the visual cue as taught by Toklu in order to provide a real-time approach of tracking 
objects in a complex environment (column 11, 22-24). 

Regarding Claim 8, Bugaj in section 6.1, General Semantics, 3 discloses 
schedule elements with begin and end time and in section 6.2 page 8, 3 Bugaj discloses 
that duration is the difference between the end time and begin time of an element, 
which reads on "An XML-based browser as defined further wherein temporal 
characteristics include begin time, end time, and duration." 
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Regarding Claim 9, Bugaj in section 6.4, Syntax discloses an image element 
tagged as an XML media object element. 

Bugaj and Hickman are silent about wherein visual representation includes color. 
Toklu teaches color intensity inside the boundary of an object which meets "wherein 
visual representation includes color" (column 6, 39-40). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj and Hickman to include wherein visual 
representation includes color as taught by Toklu in order to provide a real-time 
approach of tracking objects in a complex environment (column 11, 22-24). 

Regarding Claim 10, Bugaj in section 6.4, Syntax discloses an image attribute 
which reads on "An XML-based browser as defined wherein visual representation 
includes shape." 

Regarding Claim 11, in section 5, General Semantics, 1-6 Bugaj disclose a 
layout section of a SMIL document including alternative layout elements embedded in a 
switch element used to determine the placement of the presentation which reads on "An 
XML-based browser wherein spatial characteristics includes position." 

Regarding Claim 12, in section 7.4 page 27 Bugaj discloses a smile document for 
the newscast presentation illustrated in figure 7.1 page 26 wherein the layout and 
temporal elements are controlled by their associated attributes in hierarchal order which 
reads on "An XML-based browser wherein the XML-based tag for the visual cue is 
nested within the XML based element that defines the associated visual element." 
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Claim 13 is the software that performs the steps described in claim 7. Thus claim 
1 3 is rejected for the same reasons as claim 7. 

Regarding Claim 14, Bugaj in section 6.1, General Semantics, 3 discloses 
schedule elements with begin and end time and in section 6.2 page 8, 3 Bugaj discloses 
that duration is the difference between the end time and begin time of an element, 
which reads on "A computer-readable medium, wherein temporal characteristics include 
begin time, end time, and duration." 

Regarding Claim 15, Bugaj in section 6.4, Syntax discloses an image element 
tagged as an XML media object element. 

Bugaj and Hickman are silent about wherein visual representation includes color. 

Toklu teaches color intensity inside the boundary of an object which meets 
"wherein visual representation includes color" (column 6, 39-40). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj and Hickman to include wherein visual 
representation includes color as taught by Toklu in order to provide a real-time 
approach of tracking objects in a complex environment (column 11, 22-24). 

Regarding Claim 16, Bugaj in section 6.4, Syntax discloses an image attribute 
which reads on "A computer-readable medium, wherein visual representation includes 
shape." 

Regarding Claim 17, in section 5, General Semantics, 1-6 Bugaj disclose a 
layout section of a SMIL document including alternative layout elements embedded in a 
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switch element used to determine the placement of the presentation which reads on "A 
computer-readable medium, wherein spatial characteristics include position." 

Regarding Claim 18, in section 7.4 page 27 Bugaj discloses a smile document for 
the newscast presentation illustrated in figure 7.1 page 26 wherein the layout and 
temporal elements are controlled by their associated attributes in hierarchal order which 
reads on "A computer-readable medium according to Claim 13, wherein the XML-based 
tag for the visual cue is nested within an XML-based element that defines the 
associated visual element." 

Regarding claim 19, Bugaj in section 7.4 page 27 Bugaj discloses a smile 
document for the newscast presentation illustrated in figure 7.1 page 26 wherein the 
layout and temporal elements are controlled by their associated attributes which reads 
on "An XML-based tag, wherein the temporal characteristic of the visual cue is based on 
the temporal characteristic of the visual element to which the visual cue is associated." 

Regarding claim 20, Bugaj in section 7.4 page 27 Bugaj discloses a smile 
document for the newscast presentation illustrated in figure 7.1 page 26 wherein the 
layout and temporal elements are controlled by their associated attributes which reads 
on "An XML-based browser, wherein the temporal characteristic of the visual cue is 
based on the temporal characteristic of the multimedia element to which the visual cue 
is associated." 

Regarding claim 21, Bugaj in section 7.4 page 27 Bugaj discloses a smile 
document for the newscast presentation illustrated in figure 7.1 page 26 wherein the 
layout and temporal elements are controlled by their associated attributes which reads 
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on "A computer-readable medium, wherein the temporal characteristic of the visual cue 
is based on the temporal characteristic of the multimedia element to which the visual 
cue is associated." 

Regarding Claim 15, Bugaj in section 6.4, Syntax discloses an image element 
tagged as an XML media object element. 

Bugaj and Hickman are silent about wherein visual representation includes color. 

Toklu teaches color intensity inside the boundary of an object which meets 
"wherein visual representation includes color" (column 6, 39-40). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bugaj and Hickman to include wherein visual 
representation includes color as taught by Toklu in order to provide a real-time 
approach of tracking objects in a complex environment (column 11, 22-24). 

Conclusion 

6. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
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extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ANTHONY BANTAMOI whose telephone number is 
(571 )270-3581 . The examiner can normally be reached on Monday - Friday 8-5:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Koenig can be reached on (571) 272 7296. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Anthony Bantamoi 

Examiner 

Art Unit 2423 

/Anthony Bantamoi/ 
Examiner, Art Unit 2423 
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